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Abstract 


Coaching with live observation and immediate performance feedback is an effective means to train paraprofessionals, 
but might not always be feasible. We used a multiple baseline across participants design with six paraprofessionals who 
taught elementary students with severe disabilities to test the efficacy of two innovations designed to improve the 
feasibility of delivering feedback. We found a functional relation between delayed, video-based performance feedback and 
paraprofessional implementation fidelity of two systematic prompting strategies. Observing a colleague receive feedback 
did result in some improvement, but did not enable all paraprofessionals to meet the training criterion. These findings 
suggest that delayed, video-based feedback is an effective and feasible training tool, but only observing a colleague receive 


feedback might be insufficient. 
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Paraprofessionals play a large role in the education of stu- 
dents with severe disabilities. Paraprofessionals outnumber 
licensed special education teachers in the United States by 
a ratio of 1.1:1 (U.S. Department of Education, 2017). 
Although federal data do not disaggregate paraprofession- 
als and teachers who serve only students with severe dis- 
abilities, state and local-level data suggest that the ratio is 
even higher for this subgroup (Suter & Giangreco, 2009). 
According to one descriptive study, 99% of paraprofes- 
sionals report that they spend at least some time providing 
one-to-one instruction to students (Carter et al., 2009). For 
paraprofessional-delivered instruction to promote student 
progress, it is critically important for paraprofessionals to 
be well-trained and supported to implement evidence- 
based practices that promote student learning (Brock & 
Carter, 2013). 

Scholars have raised serious concerns about whether 
paraprofessionals are sufficiently trained and supervised in 
these instructional roles, and whether paraprofessional- 
delivered instruction is appropriately contextualized as sup- 
plemental to teacher-delivered instruction (e.g., Suter & 
Giangreco, 2009). They point to problematic cases when 
paraprofessionals deliver instruction with insufficient 
teacher oversight (Giangreco & Broer, 2005). Others have 


pointed out that paraprofessionals are rarely given the train- 
ing needed to deliver instruction effectively (Brock & Carter, 
2013; Carter et al., 2009). Scholars also suggest that in some 
cases, paraprofessional-delivered instruction might be inap- 
propriately used as a substitute for teacher-delivered instruc- 
tion, which deprives students of the opportunity to learn 
from highly qualified teachers (Giangreco et al., 2001). 
Indeed, these kinds of practices are ethically problematic 
(Giangreco et al., 2001), inconsistent with federal law 
(Individuals with Disabilities Education Improvement Act, 
2004), and unlikely to promote optimal progress for students 
with severe disabilities (Brock & Carter, 2013). 

To address these concerns, it is critically important for 
schools to only task paraprofessionals with instructional roles 
that fall within certain parameters. Specifically, paraprofes- 
sionals should only deliver instruction (a) when a teacher has 
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designed the instruction; (b) when the paraprofessional has 
been given adequate training to implement the instruction as 
designed by the teacher; (c) when teachers provide adequate 
oversight and feedback; (d) when paraprofessionals share 
student data with teachers, and teachers decide if and how to 
adjust instruction; and (e) when this instruction is supple- 
mental to teacher-delivered instruction (Uitto et al., 2016). 

Researchers have demonstrated how paraprofessionals 
can be tasked with delivering instruction and support within 
these parameters. For example, Brock et al. (2016) showed 
that special education teachers can create individualized 
plans that outline how paraprofessionals should facilitate 
peer support arrangements in middle school general educa- 
tion classrooms, and then direct and supervise paraprofes- 
sionals to implement the plans. They provided evidence that 
paraprofessional implementation of peer support arrange- 
ments improved social outcomes for students with severe 
disabilities. In another study, Wermer et al. (2017) demon- 
strated that a teacher can use coaching and feedback to 
enable a paraprofessional to elicit and promote communica- 
tion for an elementary student with autism and complex 
communication challenges. These examples illustrate that it 
is indeed possible for paraprofessionals to provide instruc- 
tion that is teacher-directed, supplemental to teacher-deliv- 
ered instruction, and effective in improving outcomes for 
students with severe disabilities. 

Despite these examples from the research literature, 
there is little evidence that similar models have been adopted 
in everyday practice (Brock & Carter, 2013). One possible 
explanation might be that these training and supervision 
models are intensive and time-consuming. Specifically, 
research-based paraprofessional training has focused heav- 
ily on repeated live observations and immediate perfor- 
mance feedback in a one-to-one format (Brock & Carter, 
2013). This approach involves a trainer visiting the class- 
room to observe attempted implementation of a practice in 
real time, and then meeting with the paraprofessional to 
share constructive feedback. This feedback focuses on 
aspects of implementation that were well-executed, and 
how others might be improved in the future. Researchers 
have recommended that performance feedback be delivered 
during or immediately after a teaching session because this 
has tended to be more effective in the research literature 
(Scheeler et al., 2016). Furthermore, coaching is often 
extended over time. In the existing literature, trainers some- 
times delivered five or more training sessions to paraprofes- 
sionals to ensure sustained implementation fidelity of a 
single practice (e.g., Leblanc et al., 2005). 

There are a number of reasons why a teacher would find 
it challenging to deliver this kind of one-to-one coaching to 
paraprofessionals. These challenges stem from a teacher’s 
limited time during the school day, which they report is a 
major barrier to training paraprofessionals (French, 2001). 
This limited time makes it difficult for teachers to (a) 


observe paraprofessionals during the school day, (b) deliver 
performance feedback immediately after an observation, (c) 
schedule multiple training sessions within a short window 
of time, and (d) replicate this one-to-one training across all 
paraprofessionals whom they supervise. Innovations are 
needed to improve the feasibility of performance feedback. 

In this study, we investigated some practical solutions 
that might mitigate or eliminate these challenges. First, we 
eliminated the need for trainers to conduct live observations 
by video recording paraprofessional implementation, pro- 
viding trainers the flexibility to observe implementation at 
a more convenient time, such as during a teacher planning 
period or before or after school. Second, we examined 
whether performance feedback could result in rapid acqui- 
sition of implementation fidelity even if delivered after a 
short delay (i.e., 2-3 school days). Third, we tested whether 
paraprofessionals would be able to acquire and maintain 
implementation fidelity given only two sessions of direct 
performance feedback. Fourth, we examined whether a 
paraprofessional might benefit from observing a colleague 
receive performance feedback. If paraprofessionals can 
improve their implementation by observing a colleague 
receive feedback (without directly receiving feedback on 
their own practice), trainers could more efficiently train 
paraprofessionals in small groups. We hypothesized that the 
observing paraprofessional would make progress by view- 
ing additional exemplars that could be generalized to her 
own implementation (Stokes & Baer, 1977). For each step, 
the observing paraprofessional would see an exemplar from 
her colleague, or the trainer point out the error and model an 
exemplar for that step. Furthermore, the trainer and the col- 
league would practice steps on which errors had been made, 
providing additional opportunities to observe correct imple- 
mentation. In addition, the observing paraprofessional 
would see the trainer distinguish whether each step was fol- 
lowed with fidelity, which could generalize to the parapro- 
fessional more accurately judging her own implementation 
fidelity. To ensure that the feedback they observed would be 
directly relevant to their own implementation, we designed 
the study so that each pair of paraprofessionals would be 
targeting very similar types of skills; specifically, all stu- 
dents were labeling discrete targets; see “Student progress” 
in the “Method” section for further description. 

Although performance feedback has been identified as 
a key active ingredient in paraprofessional training, it is 
rarely delivered in isolation of other training strategies 
(Brock et al., 2017). Researchers often pair performance 
feedback with written instructions and an implementation 
checklist (Brock & Carter, 2013). In an effort to isolate 
the effects of performance feedback, we first tested the 
effects of written instructions alone before introducing 
performance feedback. We did not anticipate that written 
instructions alone would promote implementation fidelity 
(Brock et al., 2017). We used these strategies to train 
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paraprofessionals to implement simultaneous prompting 
and least-to-most (LTM) prompting, because these two 
strategies have been well established as evidence-based 
practices (Wong et al., 2015). 

Specifically, we addressed the following research ques- 
tions: First, is there a functional relation between written 
instructions and paraprofessional implementation of two 
systematic prompting strategies? Second, is there a func- 
tional relation between delayed video-based feedback and 
paraprofessional implementation? Third, is there a func- 
tional relation between observing a colleague receiving 
feedback and paraprofessional implementation? Fourth, do 
the effects of delayed video-based feedback maintain after 
the training is over? Fifth, what progress did students who 
receive instruction from paraprofessionals make on indi- 
vidualized goals? 


Method 


Paraprofessionals and Students 


After receiving Institutional Review Board and district 
approval, we recruited six paraprofessionals and five stu- 
dents with severe disabilities. To be included, a paraprofes- 
sional must have provided one-to-one instruction to an 
elementary student with a severe disability (i.e., received 
special education services under the category of intellectual 
disability, autism, or multiple disabilities; and was eligible 
for the state’s alternate assessment). All paraprofessional 
reported that they were unfamiliar with and had not received 
training on simultaneous prompting, LTM prompting, or 
any other instructional strategy prior to the study. In addi- 
tion to providing one-to-one instruction to the target stu- 
dents, all paraprofessionals also had other responsibilities at 
other times of the school day (e.g., providing one-to-one 
instruction to other students; supporting students in general 
education classrooms; supervising students during lunch, 
recess, and transitions). Paraprofessionals and students are 
described in the following. 


Anna and Adam. Anna was a 50-year-old White female 
paraprofessional with 6 years of experience in special edu- 
cation and a bachelor’s degree in an unrelated field. Anna 
had received district-provided training on crisis prevention. 
She taught Adam, an 11-year-old White male student with 
autism spectrum disorder (ASD). Adam had a standard 
score of <50 on the Developmental Profile—Third Edition 
(Alpern, 2007). 


Belva and Brenda. Belva was a 46-year-old White female 
paraprofessional with 6 years of experience in special edu- 
cation and a high school diploma. Brenda had received 
district-provided training on crisis prevention and behav- 
ior management. She taught Brenda, a 10-year-old White 


female student with ASD. Brenda had a Global Adaptive 
Composite score of 66 on the Adaptive Behavior Assess- 
ment System—Second Edition (ABAS-II; Harrison & Oak- 
land, 2003). 


Clair and Carl. Clair was a 51-year-old White female para- 
professional with | year of experience in special education 
and a high school diploma. She had received district-pro- 
vided training on managing problem behavior, crisis preven- 
tion, and first aid. She taught Carl, a 7-year-old Black male 
student with ASD. Carl had a standard score of 32 on the 
Vineland Adaptive Behavior Scales-Third Edition (Vine- 
land-III; Sparrow et al., 2005). Carl frequently exhibited 
challenging behavior (i.e., hitting, pinching, and spitting). 


Denise. Denise was a 57-year-old White female paraprofes- 
sional with 12 years of experience in special education and 
a high school diploma. She had received district-provided 
training on managing problem behavior, crisis preven- 
tion, and first aid. Denise also worked with Carl (described 
above). 


Emily and Eric. Emily was a 53-year-old White female 
paraprofessional with 21 years of experience in special 
education and a high school diploma. She had received 
district-provided training on school safety. She taught Eric, 
a 13-year-old White male student with ASD and intellec- 
tual disability. Eric had an IQ score of 40 on the Wechsler 
Intelligence Scale for Children—Fourth Edition (WISC-IV; 
Wechsler, 2003) and a standard score of 63 on the 
Vineland-III. 


Felicia and Frank. Felicia was a 59-year-old White female 
paraprofessional with 7 years of experience in special edu- 
cation and an associate’s degree. She had received district- 
provided training on school safety. She taught Frank, a 
9-year-old White male student with multiple disabilities. 
Frank had an IQ score of 56 on the Stanford-Binet Intelli- 
gence Scales (Roid, 2003) and a standard score of 80 on the 
ABAS-II. 


Settings and Materials 


Participants were recruited from two elementary schools in 
two school districts. The first served nearly 600 students in 
rural and suburban communities, approximately 65% of 
whom were White and about one sixth received free or 
reduced-priced meals. The second served about 400 stu- 
dents in a rural area. More than 95% of the students were 
White and about two thirds received free or reduced-price 
meals. Both schools were located in the Midwest. All 
instruction was delivered in self-contained special education 
classrooms. All training was provided to paraprofessional in 
these same classrooms, where we either sat together at a 


Focus on Autism and Other Developmental Disabilities 00(0) 


cluster of desks (i.e., Anna and Belva) or at a table (Clair, 
Denise, Emily, and Felicia). Training materials included a 
6-page handout and an inexpensive computer tablet (i.e., 
Amazon Fire) with a mini-tripod. 


Dependent Measures and Recording 


We collected data on both paraprofessional implementation 
fidelity and individualized student outcomes. All data were 
collected live using a pencil-and-paper system. 


Paraprofessional implementation fidelity. We collected data on 
implementation of simultaneous prompting and LTM 
prompting. Simultaneous prompting involves delivering a 
controlling prompt (i.e., the least intrusive prompt that 
results in the student performing the skill correctly) imme- 
diately after delivering a cue or task direction. In contrast, 
LTM prompting involves giving an opportunity to respond 
independently, and then providing increasing levels of 
assistance until the student performs the skill correctly. 

We focused on two dimensions of fidelity: adherence to 
the protocol and quality of implementation (O’ Donnell, 
2008). Both dimensions were measured at every observa- 
tion. The primary dimension was adherence to the protocol. 
First, we told the paraprofessional to use simultaneous 
prompting. Then we used a checklist to measure whether 
the paraprofessional adhered to the implementation of each 
step associated with simultaneous prompting, including (a) 
delivering a cue or task direction; (b) immediately provid- 
ing a controlling prompt; and (c) providing specific praise 
after a correct response, or repeating the controlling prompt 
after an incorrect response. Each step was scored as imple- 
mented or not implemented. After we scored three trials, we 
directed the paraprofessional to teach using LTM prompting 
as if the student had already worked on the skill for several 
days. Then we measured the degree to which the parapro- 
fessional adhered to the implementation steps associated 
with LTM prompting, including (a) delivering a cue or task 
direction; (b) providing a 3 to 5 s interval for an indepen- 
dent student response; and (c) providing specific praise 
after a correct response, or providing prompts of increasing 
intensity after an incorrect response. For each prompting 
procedure, we calculated the percentage of steps imple- 
mented correctly out of the total number of steps across the 
three trials. In our previous work, we have found data from 
three trials provides a representative sample of fidelity 
(Brock, Seaman, & Downing, 2017). For both procedures, 
our goal was to train paraprofessionals to at least 80% fidel- 
ity across three consecutive sessions. 

Quality of implementation was treated as a secondary, 
descriptive dimension of implementation fidelity. At the end 
of each observation, we scored six items on a 4-point Likert- 
type scale with three being the highest quality and zero being 
the poorest quality. Items focused on pacing of instruction, 


consistency of delivering the same cue or task direction, 
immediacy of prompting after an error, verbal praise, imme- 
diacy of reinforcement after a correct response, and overall 
quality of implementation (see Table 1). This is an instru- 
ment that was constructed by our research team. We selected 
areas of focus based on our anecdotal notes from observa- 
tions in previous studies, and we based the specific quality 
indicators on evidence-based recommendations for imple- 
mentation (1.e., Neitzel & Wolery, 2009). We computed an 
average score across items as a measure of overall quality. 
Higher scores represented higher quality instruction. 


Student progress. We tracked student progress as a secondary 
dependent variable. Student learning targets were selected 
from each student’s Individualized Education Program (IEP) 
in collaboration with the student’s teacher of record. Adam 
was labeling the names and values of four real coins (i.e., 
penny, nickel, dime, and quarter). Brenda was defining 10 
vocabulary words printed on flashcards from the general 
education science curriculum. These words included carni- 
vore, herbivore, omnivore, decomposer, producer, con- 
sumer, food chain, food web, predator, and prey. Carl was 
identifying the letters in his own name printed on laminated 
pieces of paper (with Clair) and identifying colors printed on 
laminated paper (with Denise). The colors were red, blue, 
and yellow. Eric and Frank were each reading 10 high-fre- 
quency sight words on flashcards. Eric’s words included 
their, other, many, been, called, your, have, over, sound, and 
where. Frank’s words included these, down, made, with, 
what, water, were, place, years, and little. 


Experimental Design and Procedures 


We used a multiple baseline across participants design (Gast 
& Ledford, 2014) to test the efficacy of (a) written direc- 
tions relative to no training; (b) observing feedback relative 
to written directions; (c) direct video-based feedback rela- 
tive to written directions; and (d) direct video-based feed- 
back relative to observing feedback. We required a minimum 
of three data points in the no training condition, and five 
data points in all other conditions. Paraprofessionals were 
paired so that they would begin the video-based feedback 
session simultaneously and one could receive direct feed- 
back while the other observed. We changed conditions for a 
given pair of paraprofessionals when the minimum number 
of points were collected for those paraprofessionals, trends 
were flat across all paraprofessionals, and we were able to 
schedule any training required in the subsequent condition. 
We did not require any level of minimum or criterion-level 
performance before changing conditions. 


No training. We did not deliver any training to paraprofes- 
sionals in this condition. We simply measured the depen- 
dent variables as described in the “Dependent Measures and 
Recording” section. 
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Table I. Items Scored for Measure of Quality of Implementation Fidelity. 


Item focus Score Descriptor 
Pacing 0 Virtually no pause (<| s between all trials) or long pause (>5 s) between all trials 
I Brief pause for less than half of trials 
2 Brief pause for at least half, but not all trials 
3 Brief pause (I—5 s) between all trials 
Consistency of 0 Topography differed across multiple trials 
cue | Topography differed for one trial 
2 Identical in topography, but differed in presentation (e.g., verbal directions with different 
wording) 
3 Topography and presentation consistent across all trials 
Immediacy of 0 No prompt is delivered 
prompt after | Prompt is very delayed after error is apparent (>3 s) 
error 2 Prompt is somewhat delayed after error is apparent (1-3 s) 
3 Prompt is delivered immediate after error is apparent (=| s) 
Verbal praise 0 Some praise has negative tone that sounds insincere and disingenuous 
I All praise has a neutral tone that is neither negative nor positive 
2 Tone is inconsistent; sometimes positive and sometimes neutral 
3 All praise has a positive tone that is sincere and genuine 
Immediacy of 0 Reinforcement is never delivered 
reinforcement I Reinforcement is very delayed (>3 s) after a correct response 
2 Reinforcement is somewhat delayed (I—3 s) after a correct response 
3 Reinforcement is immediate (=| s) after a correct response 
Overall 0 Poor: There is one significant problem with implementation quality (e.g., negative tone, poorly 
impression chosen prompt, ineffective reinforcer) 
| Fair: Overall, there were no significant problems with implementation quality, but there are two 
or more minor issues that could be improved (e.g., sometimes reinforcement is slightly delayed, 
wording of task direction is sometimes slightly different) 
2 Good: Overall, there were no significant problems with implementation quality, but there is one 
minor issue that could be improved 
3 Excellent: Overall, there are no significant or minor issues related to quality of implementation 


that could be improved 


Note. All items were scored on a 4-point scale with 0 reflecting the poorest quality and 3 reflecting the highest quality. Scores across items were averaged 


to compute an overall score. 


Written directions. We provided paraprofessionals with a 
6-page handout. The first page defined simultaneous 
prompting, and explained when and why it should be used. 
The second page defined a controlling prompt, explained 
how to select a controlling prompt based on the skill and 
student, and provided examples of controlling prompts. The 
third page defined LTM prompting, and explained when and 
why it should be used. The fourth page explained how to 
select levels in a prompting hierarchy based on the skill and 
student, and provided examples of prompting hierarchies. 
The fifth and sixth pages were detailed implementation 
checklists for simultaneous prompting and LTM prompting. 
We instructed paraprofessionals to read the handouts and do 
their best to implement the procedures. 


Delayed video-based feedback. There were two different 
ways delayed video-based feedback was provided: Direct 
feedback (i.e., feedback on one’s own performance 2—3 days 
after implementation) and indirect feedback (i.e., observing 


someone else receive feedback without receiving any feed- 
back on one’s own performance). Within each pair of para- 
professionals, one paraprofessional was selected randomly 
to receive direct feedback. The selected paraprofessional 
used a computer tablet with an attached mini-tripod to record 
her own instruction. Two to three school days after the video 
was recorded, the first author provided feedback to the 
selected paraprofessional on her performance (i.e., direct 
feedback) while the second paraprofessional observed the 
feedback session (i.e., indirect feedback). The second para- 
professional observed all aspects of the feedback session, 
and was only permitted to ask questions that related directly 
to the colleague’s video. Feedback sessions involved the 
first author (a) reviewing all implementation steps prior to 
watching the video; (b) watching the video and providing 
specific praise and/or corrective feedback for each imple- 
mentation step; and (c) engaging in role-play (with only the 
paraprofessional receiving direct feedback) to confirm that 
the paraprofessional could correctly implement each step. 
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Feedback was not scripted. On average, feedback sessions 
lasted 29 min (range = 26-36). Each paraprofessional 
received two sessions of direct feedback. After both para- 
professionals demonstrated stable patterns of responding 
(i.e., flat trends), the second paraprofessional transitioned to 
the direct feedback phase. 


Maintenance. We terminated all feedback sessions, but 
allowed paraprofessionals to keep and refer to the written 
directions. 


Procedural Fidelity 


For the written directions condition, all paraprofessionals 
reported that they had read all written materials. For the 
video-based performance feedback condition, the trainer 
completed a 16-item checklist during each feedback ses- 
sion. An observer also completed this checklist during 
83.3% of sessions. Steps were followed with 100% fidelity 
across all sessions, and agreement was 100% between the 
trainer and the observer. 


Observer Training and Reliability 


The second author was the primary data collector, and the 
third author and one other observer collected secondary 
data to measure reliability. All observers were doctoral stu- 
dents in special education and were trained by the first 
author, a faculty member in special education. Before col- 
lecting data, observers were required to (a) score 100% ona 
written test of coding definitions and rules, (b) achieve 95% 
agreement with the first author when coding a training 
video, and (c) achieve 95% agreement with the first author 
in a live setting. 

Two observers collected data during 32% of all observa- 
tions, balanced across participants and conditions. We com- 
puted point-by-point agreement on each behavior. Total 
agreements (i.e., identical codes for both observers) were 
divided by opportunities for agreement. Overall agreement 
across all paraprofessional behaviors was 98.7% (range = 
80.0%—100.0%). Total agreement across all student behav- 
iors was 99.5% (range = 83.3% 100.0%). 


Social Validity 


We used a 10-item paper-pencil questionnaire to measure 
paraprofessional perceptions of their own competence, the 
training, and their likelihood to use systematic instruction 
and participate in similar training opportunities in the 
future. Responses were provided on a 5-point Likert-type 
scale. In addition to these forced choice items, we invited 
paraprofessionals to write what they liked most about the 
training, and what they thought could be improved. 


Results 


Paraprofessional Adherence to Steps 


Adherence data for simultaneous prompting and LTM 
prompting are displayed in Figure | and described below 
for each participant in terms of trend, level, and variability. 
No basic effects were demonstrated for written directions 
alone. Observed feedback resulted in a change in data pat- 
terns across three participants, but did not consistently pro- 
duce the desired effect of sustained criterion-level adherence 
(i.e., three consecutive data points =80%). In one case, a 
paraprofessional did reach criterion-level adherence with 
observed feedback. In all other cases, direct video-based 
feedback enabled participants to rapidly acquire criterion- 
level adherence for both procedures, demonstrating a func- 
tional relation. Effects maintained for all participants after 
all feedback was withdrawn. 


Anna. During the no training condition, simultaneous 
prompting data began at 0%, increased to 50%, and then 
remained flat at 50%. LTM prompting data were flat and 
between 40% and 50%. During the written instructions con- 
dition, simultaneous prompting slowly trended upward to 
peak at 75% and then decreased to 63%. LTM prompting 
data were more variable compared with no training, and 
lower in level. No effects were demonstrated for written 
directions. During the direct feedback condition, simultane- 
ous prompting data immediately increased in level and 
trended to 100% and remained at that level. LTM prompting 
data immediately increased to 100% and maintained at that 
level. Therefore, basic effects were demonstrated for direct 
feedback. Data for both procedures remained at 100% 
throughout the maintenance condition. 


Belva. During the no training condition, data for both pro- 
cedures had a flat trend and ranged 25% to 40%. During 
the written directions condition, data for both procedures 
were initially variable. Simultaneous prompting data sta- 
bilized at 63%, and LTM prompting data remained vari- 
able between 25% and 67%. During the observed feedback 
condition, simultaneous prompting data initially decreased, 
trended upward to 88%, and then trended downward to 
50%. LTM prompting data immediately increased in level 
to 83%, and then fluctuated between 60% and 100%. 
Although there was a clear shift in data patterns, and Belva 
did demonstrate criterion-level implementation fidelity 
for both procedures, this level of fidelity was not sustained 
across three consecutive sessions. During the direct feed- 
back condition, data for both procedures immediately 
increased to 100% and remained at that level; therefore, 
basic effects were demonstrated for direct feedback. 
Data remained at 100% throughout the maintenance 
condition. 
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Figure |. Paraprofessional adherence to steps for simultaneous prompting (open circles) and least-to-most prompting (closed 


triangles). Observed feedback involved watching a colleague receive feedback on her performance; direct feedback involved receiving 
feedback on one’s own performance. 
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Clair. During the no training condition, simultaneous 
prompting data were flat at 0% with the exception of one 
data point at 50%. LTM data were initially flat at 50%, 
trended downward to 0%, and then back up to 50%. During 
the written directions condition, simultaneous prompting 
data were more variable, but four of the seven points were 
still 0%. LTM prompting data were flat, with all data 
between 40% and 50%. No basic effects were demonstrated 
for written directions. During the direct feedback condition, 
data for both conditions increased immediately, and were 
somewhat variable between 83% and 100%; therefore, 
basic effects were demonstrated for direct feedback. Data 
remained at 100% throughout the maintenance condition. 


Denise. During the no training condition, simultaneous 
prompting data trended from 63% down to 0%. LTM 
prompting data trended toward 50% and remained flat 
between 40% and 50%. During the written directions condi- 
tion, simultaneous prompting data initially increased and 
then trended down to 0%. LTM prompting data remained 
flat at 40% to 50%. No basic effects were demonstrated for 
written directions. During the observed feedback condition, 
simultaneous prompting data increased in level, with all but 
one point between 57% and 75%. LTM data initially trended 
upward to 88% before trending back down to 50% to 63%. 
Although Denise’s performance improved relative to the 
previous condition, observed feedback did not result in sus- 
tained criterion-level implementation fidelity for either pro- 
cedure. During the direct feedback condition, data for both 
procedures immediately increased to 100%, with some 
variability; therefore, basic effects were demonstrated for 
direct feedback. Data remained at 100% throughout the 
maintenance condition. 


Emily. During the no training condition, simultaneous 
prompting data were flat at 0%, and LTM prompting data 
were flat at 50%, with some variability. These patterns were 
unchanged in the written directions condition; no basic 
effects were demonstrated. During the direct feedback con- 
dition, data immediately increased to 100% for both proce- 
dures and remained at that level; therefore, basic effects 
were demonstrated for direct feedback. Data remained at 
100% throughout the maintenance condition. 


Felicia. During the no training condition, simultaneous 
prompting data were flat at 0%, and LTM prompting data 
were highly variable between 0% and 67%. During the 
written directions condition, simultaneous prompting data 
remained unchanged, and LTM-prompting data decreased 
in variability but did not change in level; no basic effects 
were demonstrated. During the observed feedback condi- 
tion, simultaneous prompting data immediately increased 
in level to 63% and then trended to 100%. LTM prompting 
data increased slightly in level and variability (range = 


50%-71%). Both data patterns changed relative to the pre- 
vious condition, but sustained criterion-level fidelity was 
demonstrated only for simultaneous prompting. During the 
direct feedback condition, data maintained at 100% for 
simultaneous prompting, and increased to 100% and main- 
tained at that level for LTM prompting. 


Paraprofessional Implementation Quality 


Overall quality of implementation for each paraprofessional 
in each experimental condition is reported in Table 2. All 
paraprofessionals improved their quality of implementa- 
tion after observed feedback or direct feedback, mirroring 
improvements in adherence data. Three paraprofessionals 
maintained high quality after training was withdrawn. 


Student Progress 


Student progress was tracked as a secondary dependent 
variable, and is reported in Figure 2. Below, data are 
described in terms of trend, level, and variability. 

Adam’s data were variable between 0% and 50% across 
phases. No basic effects were demonstrated for any experi- 
mental condition. Brenda’s data were flat between 10% and 
20% in the no training and written directions conditions. 
They increased in level slightly during the observed feed- 
back condition. They trended upward to 90% during the 
direct feedback condition, and then to 100% during the 
maintenance condition. Basic effects were demonstrated for 
both observed and direct feedback. Carl’s data were variable 
for both of his instructional goals, with no basic effects dem- 
onstrated for any experimental condition. Eric’s data trended 
to 100% during no training, and maintained at 100% during 
the written directions condition. We selected a new learning 
target for Eric, and his data for the new target trended upward 
in the direct feedback condition and flattened in the mainte- 
nance condition. Because of the upward trend in the no train- 
ing condition and the change in learning targets, there was 
not an opportunity to demonstrate basic effects. Frank’s data 
trended upward to 70% during the no training condition, and 
were variable between 60% and 90% in all subsequent con- 
ditions. Due to the upward trend in the no training condition, 
there was not an opportunity to demonstrate basic effects. 


Social Validity 


In response to close-ended questions, paraprofessionals 
reported a high degree of satisfaction with the training 
and with the systematic teaching strategies (see Table 3). 
For open-ended written responses, paraprofessionals 
reported that the aspects of training that they liked best 
included the actual strategies being taught, receiving 
video-based feedback, and role-playing correct imple- 
mentation of the strategies. Paraprofessionals thought the 
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Table 2. Average Overall Quality of Implementation Scores for Paraprofessionals Across All Experimental Conditions. 


Paraprofessional Baseline Written directions Observed feedback Direct feedback Maintenance 
Anna 2.1 (1.0-3.0) 2.2 (1.22.3) — 2.8 (2.0-3.0) 3.0 (3.0-3.0) 
Belva 1.4 (0.0-1.7) 2.2 (1.0-3.0) 2.3 (1.3-3.0) 3.0 (2.8-3.0) 3.0 (3.0-3.0) 
Clair 1.6 (0.6-3.0) 2.6 (1.6—-3.0) — 2.7 (1.6-3.0) 2.0 (0.2-3.0) 
Denise 1.7 (0.7-2.8) 2.2 (1.0-3.0) 2.6 (1.73.0) 2.4 (1.5-3.0) 1.8 (0.0-3.0) 
Emily 2.1 (1.1-2.9) 2.7 (1.6—-3.0) — 3.0 (3.0-3.0) 3.0 (3.0-3.0) 
Felicia 1.3 (1.0-2.4) 1.9 (1.0-2.8) 2.7 (2.0-3.0) 2.8 (2.5-3.0) — 


Note. The range of overall scores across all sessions within the phase is reported in parentheses. Overall scores represent an average of the six items 


described in Table |. Scores range from 0 (poor quality) to 3 (high quality). 


training might be improved by providing feedback sooner 
and more frequently. 


Discussion 


There is strong evidence that repeated delivery of immediate 
performance feedback after live observations can improve 
practitioner implementation fidelity of evidence-based prac- 
tices (Scheeler et al., 2016), but this form of professional 
development would be difficult for teachers to replicate. In 
this study, we tested innovations that would improve the 
efficiency and feasibility of delivering performance feed- 
back. We identified a functional relation between delayed, 
video-based feedback, and practitioner implementation 
fidelity of two systematic prompting strategies. These find- 
ings extend the research literature in a number of key ways. 

First, delayed, video-based feedback is an effective 
means to enable paraprofessionals to quickly acquire and 
maintain implementation fidelity of systematic prompting 
strategies. We demonstrated improvements in both adher- 
ence to steps and quality of implementation. Although 
immediacy of performance feedback has been previously 
identified as a factor that might moderate the efficacy of 
performance feedback (Scheeler et al., 2016), our findings 
suggest that delayed, high-quality feedback can produce 
desired outcomes. Delayed video-based feedback provides 
the trainer with flexibility about the timing and location of 
feedback sessions, increasing feasibility. This flexibility 
could be helpful for both supervising teachers and adminis- 
trators who wish to provide feedback to paraprofessionals, 
but have many other competing responsibilities during the 
school day. Maintenance of effects suggests that parapro- 
fessionals can continue high-fidelity implementation with- 
out additional training. 

Second, observing a colleague receive feedback might 
improve paraprofessional implementation fidelity, but it 
does not consistently produce the desired outcome of sus- 
tained criterion-level fidelity. Of the three paraprofessionals 
who observed feedback, only one acquired criterion-level 
implementation fidelity for one of the prompting procedures 
through observed feedback alone. It is difficult to argue that 


small improvements during observed feedback might facili- 
tate more rapid progress during direct feedback, because all 
participants acquired fidelity immediately after receiving 
direct feedback regardless of whether they had previously 
observed feedback. We were surprised that observed feed- 
back did not have more potent effects, especially given 
that each pair of paraprofessionals were targeting very 
similar student skills. It seems that generalizing from a 
colleague’s teaching to one’s own teaching might have 
been a greater challenge for paraprofessionals than we 
anticipated. Specifically, paraprofessionals struggled with 
identifying their own implementation errors. Perhaps this 
was difficult because they were only exposed to a single 
exemplar of implementation (i.e., their colleague’s imple- 
mentation). Paraprofessionals might have been more suc- 
cessful at identifying their own implementation errors after 
observing multiple exemplars (Stokes & Baer, 1977). 

Third, as hypothesized, there was no evidence that writ- 
ten directions alone had an effect on paraprofessional 
implementation fidelity. This was unsurprising, given that 
meta-analyses have shown that written directions are inef- 
fective in isolation, but can make a positive contribution 
when combined with modeling and performance feedback 
(Brock et al., 2017). Our study corroborates these findings, 
because paraprofessionals made improvements only after 
they had access to both written directions and performance 
feedback. 

Fourth, mixed patterns of student progress might be 
attributed to a number of factors. One student made prog- 
ress that coincided with paraprofessional acquisition of 
criterion-level implementation fidelity. Other students 
began to demonstrate learning prior to paraprofessional 
training, suggesting that some students may respond to 
repeated instruction even if it is not evidence-based. Still 
other students did not make clear progress at any time in 
the study. In these cases, we offer several possible explana- 
tions. It is possible that the students would have benefited 
more from a different evidence-based approach (e.g., con- 
stant time delay, graduated guidance), and that the particu- 
lar practices used in this study (i.e., simultaneous prompting 
and LTM prompting) might not have been the best match 
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Figure 2. Student progress on individualized goals given paraprofessional instruction. 


for these particular students. Alternatively, instructional 
modifications such as more potent reinforcers or different 
instructional materials might have better promoted student 
progress. Students like Carl who exhibited challenging 


behavior may have made more progress if paraprofession- 
als were also trained to implement function-based behavior 
management strategies. 
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Table 3. Social Validity Questionnaire Ratings by Paraprofessional Participants. 


Category/question M SD 
Perception of skill after training 
How skilled are you in implementing simultaneous prompting? 4.83 0.41 
How skilled are you in implementing least-to-most prompting? 4.83 0.41 
How skilled are you in data collection? 4.67 0.81 
Perception of training 
How effective was the training at enabling you to implement new strategies with your student? 4.83 0.41 
How much do you think that the new strategies you learned contributed to your student’s 4.50 0.55 
progress on his or her goal? 
Likelihood of future implementation and training 
How likely would you be to continue to use the strategies that you learned in the future with 4.83 0.41 
the same student? 
How likely would you be to use the strategies that you learned in the future with a different 4.83 0.41 
student? 
How likely would you be to participate in a similar training opportunity in the future? 5.00 0.00 
How likely would you be to recommend a similar training opportunity to a colleague? 4.83 0.41 


Note. Response options: | = Not at all, 2 = Slightly, 3 = Somewhat, 4 = Moderately, 5 = Extremely. 


Fifth, highly variable performance in baseline condition 
demonstrates that untrained paraprofessionals do some- 
times implement some steps associated with prompting pro- 
cedures, but do not do so consistently or systematically 
without training. This is consistent with other studies that 
report highly variable paraprofessional implementation of 
systematic prompting procedures during a baseline condi- 
tion (e.g., Leblanc et al., 2005). 


Implications for Practice 


Findings from this study have important implications for 
practice. Teachers and administrators can provide effec- 
tive, low-intensity feedback to paraprofessionals by pro- 
viding them with a written implementation checklist, 
directing them to video record their teaching, and provid- 
ing delayed feedback at a time and place that are conve- 
nient. Teachers should follow their school’s policies 
related to video recording for the purpose of professional 
development. We used inexpensive (<US$100) generic 
computer tablets to record implementation, but there is no 
reason why one could not use any video-recording tech- 
nology that is already available in the classroom (e.g., 
camcorder, laptop with webcam). There may be some ben- 
efit to delivering training in a small-group format where 
paraprofessionals observe each other receiving feed- 
back, but trainers should also deliver feedback directly to 
all participants to ensure criterion-level implementation 
fidelity. Although delayed video-based feedback allows 
teachers to observe the video and provide feedback more 
flexibly, it is still important for administrators to ensure 
that teachers have time during the school day (e.g., a prep 
period) to deliver feedback. 


Limitations and Directions for Future Research 


There are a number of limitations to this study that high- 
light directions for future research. First, we did not mea- 
sure generalization, so it is unclear if paraprofessionals 
would have been able to implement practices with fidelity 
when targeting new students and situations. Furthermore, it 
is unclear how observed and direct feedback might differ- 
entially impact generalization. Researchers might design 
future studies that address these questions. Second, we 
demonstrated experimental control at the level of parapro- 
fessional implementation fidelity, but not at the level of 
student progress. This highlights two additional training 
needs, including (a) the need to train paraprofessionals in a 
range of evidence-based practices so that they can select 
practices that are the best match for a particular student and 
goal and (b) the need to train paraprofessionals to make 
adjustments to their instruction to optimize student prog- 
ress. Researchers might design future studies to identify 
effective ways to meet these needs. Third, because we only 
introduced performance feedback after the written direc- 
tions condition, one cannot rule out the possibility that per- 
formance feedback is effective only after written directions. 
Finally, visual analysis of single-case design data does 
involve some subjectivity, and it is possible that other 
visual analysts may make somewhat different judgments. 


Conclusion 


In this study, we demonstrated that delayed, video-based 
performance feedback is an effective and efficient means 
to enable paraprofessionals to implement evidence-based 
systematic prompting procedures. In contrast with previous 
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recommendations to conduct live observations and deliver 
immediate feedback, this approach provides increased flex- 
ibility to teachers who are responsible for training and 
supervising paraprofessionals. This finding contributes to 
developing more feasible and effective models of parapro- 
fessional training that have the potential to improve out- 
comes for students with severe disabilities. 
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